(1) Field of the Invention
The present invention relates to methods and devices for coding moving pictures and methods and devices for decoding the moving pictures, and more particularly to a method and a device for coding moving pictures (hereinafter, referred to as a “moving picture coding method” and a “moving picture coding device”) and a method and a device for decoding the moving pictures (hereinafter, referred to as a “moving picture decoding method” and a “moving picture decoding device”) which can reproduce the moving pictures by both of skipped display and consecutive display.
(2) Description of the Related Art
In the field of compression coding technologies for moving pictures, a Moving Picture Experts Group (MPEG) is known. In recent years, MPEG-2, MPEG-4, MPEG-4AVC, and the like have been widely used as main standards defined by the MPEG. The compression technologies of such MPEG standards are mainly characterized in that a difference between a picture to be coded (hereinafter, referred to as a “to-be-coded picture”) and a picture which has been coded and reconstructed (hereinafter, referred to as a “reference picture”) is applied with Discrete Cosine Transform (DCT) and quantization, and then variable length coding (hereinafter, regarding the MPEG compression technologies, refer to “H.264/AVC Kyokasho (H.264/AVC Textbook)”, Sakae Okubo et al., Kabushikikaisha Impress, 2004).
FIG. 1 is a diagram for explaining a coding method performed by a conventional MPEG coding device.
FIG. 1 shows an example of the case where two pictures are coded within a time period T. In FIG. 1, a horizontal axis represents a time, and numerals 1 to 4 are picture numbers assigned to pictures. Pictures assigned with the picture number 1 to 4 (hereinafter, referred to as “pictures 1 to 4”) are coded in an order of the picture numbers.
In addition, in FIG. 1, the pictures 1 to 4 have reference relationships shown by arrows of FIG. 1 as relationships between to-be-coded pictures and reference pictures. Here, it is shown that there are respective reference relationships between the pictures 1 and 2, between the pictures 2 and 3, and between the pictures 3 and 4.
In FIG. 1, a to-be-coded picture is coded with reference to a reference picture which is designated by such a reference relationship.
FIGS. 2 and 3 are diagrams for explaining a decoding method performed by a conventional MPEG decoding device.
FIG. 2 shows an example of the case where consecutive display is performed on coded data by the MPEG decoding device on the condition shown in FIG. 1 (condition that two pictures are coded within the time period T).
FIG. 3 shows an example of the case where skipped display is performed on coded data by the MPEG decoding device on the condition shown in FIG. 1 (condition that two pictures are coded within the time period T).
In FIGS. 2 and 3, in the same manner as FIG. 1, a horizontal axis represents a time, and numerals 1, 2, and 3 are picture numbers assigned to pictures. Here, it is shown that the pictures of FIGS. 2 and 3 assigned with the same picture numbers as FIG. 1 are pictures which have been coded in FIG. 1 and are to be decoded (namely, reproduced) in FIGS. 2 and 3. Here, the MPEG decoding device of FIGS. 2 and 3 is a decoding device which decodes only one picture at maximum within the time period T.
As shown in FIG. 2, in the case of the consecutive display, the pictures are decoded and displayed sequentially in the order of the picture numbers starting from the picture 1. For example, when the time period T is set to one second in the MPEG decoding device, the consecutive display results in reproducing moving pictures at a low speed (hereinafter, referred to also as “low-speed reproduction”).
On the other hand, as shown in FIG. 3, in the case of the skipped display, the picture 1 is displayed during the first time period T and then the picture 3 is displayed during the following time period T. In other words, the pictures 1 and 3 are displayed, by skipping display of the picture 2. For example, when the time period T is set to one second in the MPEG decoding device, the skipped display results in reproducing moving pictures at a constant speed (hereinafter, referred to also as “constant-speed reproduction”).
However, in order to achieve the skipped display of FIG. 3 by skipping display of the picture 2, decoding of the picture 2 should not be skipped for the skipped display. This is because, as obvious from the reference relationship shown in FIG. 1, the picture 3 has a reference relationship with the picture 2. Therefore, even the picture 2 needs to be decoded to display the picture 3. In short, there is a problem that even a picture which does not need to be displayed is to be decoded when the picture has a reference relationship with other picture to be displayed.
Therefore, the skipped display is difficult for such a device which decodes only one picture in coded data per the time period T, when a plurality of pictures coded during the time period T in the coded data have reference relationships among them.
In order to address the above problem, a method have been proposed for achieving skipped display without decoding pictures which do not need to be displayed (refer to Japanese Unexamined Patent Application Publication No. 2003-299103 (hereinafter, referred to as “Patent Reference 1”), for example.) FIG. 4 is a diagram for explaining a coding method by which the decoding device disclosed in Patent Reference 1 can eventually achieve the skipped display without decoding pictures which do not need to be reproduced.
In FIG. 4, there are reference relationships among pictures, as shown by arrows. Here, it is shown that there are localized reference relationships between the pictures 1 and 2, between the pictures 1 and 3, and between the pictures 3 and 4.
Patent Reference 1 achieves the skipped display by coding the pictures according to the reference relationships shown by the arrow of FIG. 4, and then decoding only the pictures 1 and 3 while skipping decoding of the picture 2.
In the meanwhile, one example of such devices requiring both of the skipped display and the consecutive display is video cameras capable of recording image at a high speed (hereinafter, referred to as “high-speed recording”). Using, as one example, constant-speed reproduction and slow-speed reproduction of image recorded by the high-speed recording of the video cameras, the above description is explained below in more detail.
FIG. 5 is a diagram showing one example where the video camera capable of the high-speed recording records images.
In FIG. 5, it is shown, as one example, that firstly two images (pictures) are recorded within one second in a normal recording mode and then six images (pictures) are recorded within following one second in a high-speed recording mode. This means that two pictures are coded in one second in the normal recording mode, and six pictures are coded in one second in the high-speed recording mode. It should be noted that numerals in FIG. 5 represent picture numbers assigned to the pictures and also represent an order of recording the pictures by the video camera, namely, an order of coding the pictures.
FIGS. 6 and 7 are diagrams showing examples of reproducing the images (pictures) recorded by the video camera in FIG. 5. Here, a MPEG decoding device regarding FIGS. 6 and 7 is a decoding device which decodes only two pictures at maximum within one second.
FIG. 6 shows an example of the slow-speed reproduction achieved by consecutive display in the case of a reproduction mode performing consecutive display. As shown in FIG. 6, in the case of the consecutive display, the images recorded by the video camera during a period of the high-speed recording are decoded and reproduced consecutively in an order starting from the picture 2. As a result, the display (view) becomes slow-speed reproduction. On the other hand, FIG. 7 shows an example of the constant-speed reproduction achieved by skipped display in the case of the reproduction mode performing skipped display. Since the MPEG decoding device of FIG. 7 is a decoding device which decodes only two pictures per second, the constant-speed reproduction is achieved by decoding two pictures 2 and 5 in one second, as shown in FIG. 7. Here, the constant-speed reproduction is one example of the reproduction mode performing the skipped display.
Next, methods for coding and decoding pictures recorded by the video camera are described.
FIG. 8 is a diagram for explaining a conventional coding method performed by the video camera. FIG. 8 corresponds to the above-described case of FIG. 1. FIG. 9 is a diagram for explaining a conventional decoding method performed by the video camera. FIG. 9 corresponds to the above-described skipped display of FIG. 7.
In FIG. 8, two pictures are coded per second in the normal recording mode, and six pictures are coded per second in the high-speed recording mode. Here, there are reference relationships among the pictures, as shown by arrows of FIG. 8. In more detail, in FIG. 8, each of to-be-coded pictures is coded with reference to a reference picture which is designated by a reference relationship, such as a reference relationship between the pictures 0 and 1, between the pictures 1 and 2, between the pictures 2 and 3, between the pictures 3 and 4, or between the pictures 4 and 5.
However, in order to achieve the skipped display of FIG. 9 by skipping display of the pictures 3 and 4 or the pictures 6 and 7, decoding of the pictures 3 and 4 or the pictures 6 and 7 which do not need to be displayed should not be skipped for the skipped display. This is because, for the same reason as FIG. 2, as obvious from the reference relationship shown in FIG. 8, pictures which do not need to be displayed are to be decoded when the pictures have reference relationships with other pictures to be displayed.
FIG. 10 is a diagram for explaining the coding method performed by the video camera of Patent Reference 1. FIG. 10 corresponds to the above-described case of FIG. 4. FIG. 11 is a diagram for explaining the decoding method of the video camera of Patent Reference 1. FIG. 11 corresponds to the above-described skipped display of FIG. 7.
In FIG. 10, two pictures are coded per second in a normal recording mode, and six pictures are coded per second in a high-speed recording mode. Here, there are localized reference relationships among the pictures shown by arrows of FIG. 10. In more detail, in FIG. 10, each of to-be-coded pictures are coded with reference to a reference picture which is designated by a localized reference relationship, such as a localized reference relationship between the pictures 0 and 1, between the pictures 1 and 2, between the pictures 2 and 5, between the pictures 3 and 6, or between the pictures 4 and 8.
Thereby, in FIG. 11, it is possible to skip decoding of the pictures 3 and 4 or the pictures 6 and 7 which do not need in the skipped display, thereby achieving the constant-speed reproduction in which only the pictures 2 and 5 are displayed as the skipped display. This makes it possible to achieve the skipped display for constant-speed reproduction even in the decoding device having insufficient processing capability which decodes only a specific number of pictures per a predetermined time period, for example, two pictures per second.